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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including tine fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
01/08/2009 has been entered. 

Response to Amendment 

2. The amendment filed on 1 1/26/2008 has been entered. 

Response to Arguments 

3. Applicant's arguments filed 1 1/26/2008 have been fully considered but they are 
not persuasive in view of the newly discovered Shih US Patent No. 6,552,722. Shih 
teaches a virtual reality system that uses object constraints to control the visual 
interaction between the virtual objects and teaches the user may add the constraints 
while interacting with the virtual environment. Geometric constraints are described at 
column 37 line 17 to column 39 line 11. 

Applicant argues at pages 9-1 1 that Kitamura does not teach: 

constraining shape input means for specifying three-dimensional coordinates 
respectively of points on a constraining plane to indicate at the same time the shape 
and position of the constraining plane in said real space. 
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However, Shih teaches specifying three-dimensional geometric coordinates of 
points on a constraining plane to indicate at the same time both the shape and position 
of the constraining plane on a virtual object in virtual space with regard to allowing the 
user to interactively add constraints to the three-dimensional virtual objects, column 38 
lines 9-23. 

In view of Shih it would have been obvious to one of ordinary skill in the art at the 
time of applicants invention to modify Kitamura to include the claimed constraining 
shape input means because this will allow the user to interactively add constraints to the 
virtual objects in the real space which reduces the amount of precising required when 
the modeling of the augmented world is first produced, see applicants paragraph [0003] 
and Shih. 



Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 1-3 and 5-7 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

Claims 1-3: 

Applicants specification at paragraph 87 states the program itself may be the 
means, thus, the means of means plus function claims 1-3 is software only. 
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Claims 5-7: 

Supreme Court precedent^ and recent Federal Circuit decisions^ indicate that a 
statutory "process" under 35 U.S.C. 101 must (1) be tied to another statutory category 
(such as a particular apparatus), or (2) transform underlying subject matter (such as an 
article or material) to a different state or thing. While the instant claim(s) recite a series 
of steps or acts to be performed, the claim(s) neither transform underlying subject 
matter nor positively tie to another statutory category that accomplishes the claimed 
method steps, and therefore do not qualify as a statutory process. 

These method claims are not tied to a specific machine or apparatus and do not 
claim a physical transformation of an article or thing. In re Bilski, 88 USPQ2d 1385 
(Fed. Cir. 2008). Even the image capturing step and the constraining shape input step 
are not tied to a machine since these steps are merely data gathering steps. Similarly 
the virtual object placement shape storage step is merely a data storage step. The 
virtual image generating step, synthesizing step, virtual object operating step, 
constraining shape generating step, and operation aiding step are data generating 
steps. 



^ Diamond v. Diehr, 450 U.S. 175, 184 (1981); Parker v. Hook, 437 U.S. 584, 588 n.9 (1978); Gottschalk 
V. Benson, 409 U.S. 63, 70 (1972); Cochrane v. Deener, 94 U.S. 780, 787-88 (1876). 
^ In re Bilski, 88 USPQ2d 1385 (Fed. Cir. 2008). 
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Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl^ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 

USPQ 459 (1966), that are applied for establishing a background for determining 

obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

This application currently names joint inventors. In considering patentability of 

the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 

the various claims was commonly owned at the time any inventions covered therein 

were made absent any evidence to the contrary. Applicant is advised of the obligation 

under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 

not commonly owned at the time a later invention was made in order for the examiner to 

consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 

prior art under 35 U.S.C. 103(a). 
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7. Claims 1-3 and 5-8 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Yoshifumi Kitamura and Fumio Kishino, Consolidated Manipulation of Virtual and 
Real Objects, September 1997, Proceedings of the ACM symposium on Virtual reality 
software and technology, pages 133-138, in viewof Shih, US Patent No. 6,552,722. 

Kitamura teaches an augmented reality system that uses object constraints to 
control the visual interaction between the virtual objects and the real objects. 

Shih teaches a virtual reality system that uses object constraints to control the 
visual interaction between the virtual objects and teaches the user may add the 
constraints while interacting with the virtual environment. Geometric constraints are 
described at column 37 line 17 to column 39 line 1 1 . 

Column 38 lines 9-23 states: 

Constraints can also be placed interactively by using the haptic 
interactive device 10, with the user feeling a force feedback effect when 
doing so. Thus the user can place a geometric constraint at or within a 
virtual object 26 using the haptic interactive device 10. The user then uses 
the haptic interactive device 10 to control a virtual tool 28 to modify the 
virtual object 26, with the movement of the virtual tool 28 limited by the 
geometric constraint. In another embodiment, the user makes a constraint, 
such as by using the virtual tool 28 to inscribe a slight groove in the virtual 
surface 25 of a virtual object 26. The user then uses this groove as a 
geometric constraint, when using the virtual tool 28 to enlarge the groove 
to make a larger indentation in the virtual surface 25 based on the shape 
of the groove. 



Shih also discusses using a real world object in the virtual world at column 8 lines 

38-41 which states: 

In one embodiment, the haptic virtual environment contains a virtual 
object 26 that is model of a real world object that a user is creating in the 
virtual environment. 
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A detailed analysis of the claims follows. 
Claim 1: 

Kitamura teaches Claim 1 . (currently amended): An information processing 
device for aiding control operations relating to controlling the position and orientation of 

a virtual object, said device comprising: 

image-capturing means for capturing a real object in real space into a taken 
image (Kitamura: The measurements by the user in three dimensional real space, the 
device for obtaining the 3-D shape by using a range image, and the device for obtaining 
the 3-D shape by using multiple cameras captures a real image in real space. The 
three paragraphs found in section 2 on page 134 of Kitamura teaches the user 
measuring the real objects, a device measuring the real objects with range in response 
to the user, or a device measuring the real objects with multiple cameras range in 
response to the user. The claim does not limit the form of the real image, thus, 
Kitamura teaches this limitation. Note applicants paragraph [0024] does not restrict the 
image capturing unit to a particular type); 

virtual image generation means for generating a virtual image of a virtual object 
according to the position and orientation of said image capturing means (Kitamura: 
Kitamura discusses in section 2 in the first paragraph using conventional modeling 

software after precisely measuring the size or length of the real object by hand or by the 
devices which is an generation unit capable of generating three-dimensional positional 
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information. Sections 2, 4 and 5 discuss generating a virtual image(s) of a virtual 
object(s). Section 5 discusses virtual and real objects.); 

synthesizing means for synthesizing the virtual image generated by said virtual 
image generation means with the taken image generated by said image-capturing 
means (Kitamura: Sections 2, 4, 5, and 6 discusses mixing the virtual and real world 
images of virtual and real world objects.); 

virtual object placement shape storage means for storing the shape of the virtual 
object (Kitamura: Sections 2, 4, 5, and 6 discusses mixing the virtual and real world 
images of virtual and real world objects which requires storing the shape of both of the 
virtual and real world objects.); 

virtual object operating means for performing six-degree-of-freedom 
operations of said virtual object (Kitamura: In view of page 135 first paragraph under 
the heading of 5 Consolidated Manipulation Environment of Kitamura the same 6 DOF 
tracker device is used to control the position of a virtual world object(s) and to control 
the position of a constraining real world object(s) which is similar to applicants system 
where stylus 1060 is used to control the virtual world object(s) and to control the 
location of the constraining real world object(s), see applicants paragraphs [0028], 
[0030], and [0041]. ); 
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Kitamura does not fully teach: 

constraining shape input means for specifying three-dimensional coordinates 
respectively of points on a constraining plane to indicate at the same time the shape 
and position of the constraining plane in said real space; 

Kitamura further teaches: 

constraining shape generating means for generating a constraining shape based 
on at least one said constraining plane (Kitamura: Sections 3, 4, 5, and 6 discusses 
constraining shapes and introducing those shapes into the augmented reality world and 
figures 2, 4, and 5 illustrate and sections 5.1-5.4 discuss constraining shapes based on 
at least one constraining plane.); and 

operation aiding means for generating restrictions on the operations performed 
by said virtual object operating means based on the shape stored in said virtual object 
placement shape storage means and the constraining shape generated by said 
constraining shape generating means (Kitamura: Sections 5.1 to 5.4 discusses 
manipulation of the virtual world object based upon constraint conditions based on the 
shape of the real world object in response to the user using the 6 DOF tracker device. 
The constraining shape generated from the three-dimensional position information 
constrains the interaction of the virtual world object with the real world object, see 
sections 2, 4 to 5.4. The introduction on page 133 second full paragraph discusses 
augmented reality which synthesizes a virtual object with a real object. Section 5.1 
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discusses after movement is detected by the 6D0F manipulator the virtual object is 
moved according to the constraints. Figures 2, 4, and 5 illustrate and sections 5.1-5.4 
discuss constraining shapes based on at least one constraining plane). 

As stated above Kitamura does not fully teach: 

constraining shape input means for specifying three-dimensional coordinates 
respectively of points on a constraining plane to indicate at the same time the shape 
and position of the constraining plane in said real space. 

However, Shih teaches specifying three-dimensional geometric coordinates of 
points on a constraining plane to indicate at the same time both the shape and position 
of the constraining plane on a virtual object in virtual space with regard to allowing the 
user to interactively add constraints to the three-dimensional virtual objects, column 38 
lines 9-23. 

It should be noted that Kitamura's use of the 6 DOF tracker device to control the 
position of the real world object suggests using the same 6 DOF tracker device to input 
the constraining shape of the real world object since this would require less input 
devices for the user to use and learn how to use. Additionally section 2 at lines 7-14 
states "To bring an object that already exist in the real world into a computer-generated 
virtual world, it is necessary to construct accurate shape representation of the real 
object in a computer system. A traditional method for this is to use conventional 
modeling software after precisely measuring the size or length of the real object by 
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hand." which suggests using a computer input device to measure the real object in 
order to have accurate shape representation of the real object's constraining shape. 

In view of Shih it would have been obvious to one of ordinary skill in the art at the 
time of applicants invention to modify Kitamura to include the claimed constraining 
shape input means because this will allow the user to interactively add constraints to the 
virtual objects in the real space which reduces the amount of precision required when 
the modeling of the augmented world is first produced, see applicants paragraph [0003], 
and because this will allow the user to add or modify constraints when the user is 
interacting with the virtual objects, see Shih column 38 lines 9-23. 

Claim 2: 

Claim 2. (previously presented): An information processing device according 
to Claim 1 , wherein the constraining shape is defined by polygons (Kitamura: Figure 2's 
Cubes are formed with polygons.) and the apexes of the polygons are at positions 
inputted by the user (Kitamura: As discussed in the a setting unit above it would have 
been obvious to use the 6 DOF tracker device to input the constraining shape of the 
real world object which would input the apexes of the constraining shape of the 
polygon.) or the constraining shape is a plane passing through the positions inputted by 
the user (Kitamura: A real sensed surface is a planar real world object whose position 
is inputted by the user in the setting unit. As discussed in the a setting unit above it 
would have been obvious to use the 6 DOF tracker device to input the constraining 
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shape of the real world object which would input plane for a surface. Also section 
seciont 5.1 on page 136 discusses with regard to figure 2 a planar constraint.).. 

Claim 3: 

Claim 3. (currently amended): An information processing device according to 
Claim 1, wherein said operating aiding means performs at least one of the following 
operations in performing an operation controlling the position and orientation of the 
virtual object: 

a translation operation for causing translational movement of the virtual object 
based on the constraining shape (Page 137 column 1 lines 1-3 teaches translation 
when the virtual object collides with the real surface.); and/or 

a rotation operation for rotating the virtual object on an axis which is a normal 
vector at a plane where the constraining shape and the virtual object come into contact 
(Page 137 column 1 lines 1-3 teaches rotation when the virtual object collides with the 
real surface.) (Page 137 column 1 lines 1-3 also teaches translation and rotation when 
the virtual object collides with the real surface.). (For the added "and/" note page 137 
column 1 lines 1-3 teaches translation and rotation.). 

Claims 5-7: 

These claims are method claim versions of means plus function device claims 1- 
3 and claim 5-7 are rejected for the same reasons given for claims 1-3. 
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Claim 8: 

This claim is a computer readable medium claim version of device claim 1 and 
method claim 5 and claim 8 is rejected for the reasons given for device claim 1 and 
method claim 5. Additionally section 2 of Kitamura discusses computer systems 
performing the augmented reality which computer systems inherently have a compute 
readable medium. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Handley et al, 6,792,398 is similar to Shih. Geometric constraints are described 
at column 35 line 58 to column 38 line 13. 

Column 36 lines 51-64 states: 

Constraints can also be placed interactively by using the haptic 
interactive device 10, with the user feeling a force feedback effect when 
doing so. Thus the user can place a geometric constraint at or within a 
virtual object 26 using the haptic interactive device 10. The user then uses 
the haptic interactive device 10 to control a virtual tool 28 to modify the 
virtual object 26, with the movement of the virtual tool 28 limited by the 
geometric constraint. In another embodiment, the user makes a constraint, 
such as by using the virtual tool 28 to inscribe a slight groove in the virtual 
surface 25 of a virtual object 26. The user then uses this groove as a 
geometric constraint, when using the virtual tool 28 to enlarge the groove 
to make a larger indentation in the virtual surface 25 based on the shape 
of the groove. 



9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jeffery A. Brier whose telephone number is (571) 272- 



Application/Control Number: 10/615,941 Page 14 

Art Unit: 2628 

7656. The examiner can normally be reached on M-F from 7:30 to 4:00. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, Xiao Wu 
can be reached at (571) 272-7661 . The fax phone Number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retheval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



/Jeffery A. Brier/ 

Primary Examiner, Art Unit 2628 



